Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/onnxruntime recipe #5180

Closed
wants to merge 15 commits into from

Conversation

CAMOBAP
Copy link
Contributor

@CAMOBAP CAMOBAP commented Apr 10, 2021

New recipe for OnnxRuntime/1.7.1

per #4806


  • I've read the guidelines for contributing.
  • I've followed the PEP8 style guides for Python code in the recipes.
  • I've used the latest Conan client version.
  • I've tried at least one configuration locally with the
    conan-center hook activated.

@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

recipes/onnxruntime/all/conanfile.py Outdated Show resolved Hide resolved
Comment on lines 2 to 10
"1.7.1":
sha256: e24ff9a2a21dab74e6dfa821e9a267baa10e9423ffae5f5e404e3a2949eb41e2
url: https://github.com/microsoft/onnxruntime/archive/v1.7.1.tar.gz
"1.5.3":
sha256: 40fc79f8a3126caba5fcdced260439c7b18c72214fb8f7268d87fac22098dfba
url: https://github.com/microsoft/onnxruntime/archive/v1.5.3.tar.gz
"1.2.0":
sha256: 1ed2a4303d621682c42ebaf99fb9a8ecdc386035b15925b148aa56944abc38c8
url: https://github.com/microsoft/onnxruntime/archive/v1.2.0.tar.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only one version for the first iteration please, to have quick results from CI

Comment on lines 15 to 14
tools.download(
"https://github.com/microsoft/onnxruntime/raw"
"/master/csharp/testdata/squeezenet.onnx",
"squeezenet.onnx"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not reproducible, downloads outside of source() should be avoided.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source() doesn't work for test_package for some reason

Copy link
Contributor

@elejke elejke Mar 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spacelm I think, we can create much more simple .onnx graph about ~10kb, which can be stored in the test folder in the repo, i.e. like it is done in some packages for locale checking etc

@conan-center-bot

This comment has been minimized.

@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Apr 11, 2021

@SpaceIm thanks for the review, I agree with all feedback will work to fix it

@stale
Copy link

stale bot commented May 11, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label May 11, 2021
@dryarullin
Copy link

@CAMOBAP Are you still making a recipe?

@stale stale bot removed the stale label May 12, 2021
@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented May 12, 2021

@DenisYarullin yep this is still in my TODO list. Unfortunatelly I switched to other work((

I plan to update this PR this week

@stale
Copy link

stale bot commented Jun 11, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 11, 2021
@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Jun 14, 2021

I have some progress but it's still not enough to push any update, I will push ASAP once will have something working (I hope this week)

@stale stale bot removed the stale label Jun 14, 2021
@conan-center-bot

This comment has been minimized.

@CAMOBAP CAMOBAP force-pushed the feature/onnxruntime-recipe branch from 122ceaf to 2dc3183 Compare June 28, 2021 13:56
@stale stale bot removed the stale label Sep 10, 2021
@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Sep 11, 2021

@fdgStilla thanks a lot for help with this, once #6085 will be pushed I will continue work on this one

BTW which version on OnnxRuntime have you tested with changes in #5180 (comment) it looks > 1.7.1

@conan-center-bot

This comment has been minimized.

@fdgStilla
Copy link
Contributor

@CAMOBAP I did not change the version so it retrieves https://github.com/microsoft/onnxruntime/archive/v1.7.1.tar.gz as you initially wrote it.
I found another issue with the recipe, you have to remove all constraint related to boost in the configure method, otherwise the consumer cannot use boost libraries since it is limited only to header_only.
As an example numpy does not force to use boost header only option even if it is itself a header only pacakge: https://github.com/conan-io/conan-center-index/blob/5a411fee024156126a7a82b050218a9a649f1d2c/recipes/numcpp/all/conanfile.py

@fdgStilla
Copy link
Contributor

I see that you have : #error This file was generated by a newer version of protoc in your build log, so you should update the proto package version like I did, the other changes are also necessary. I tested on both Windows version 16 and gcc version 10 (libstdc++11) with shared/static.

self._add_definition("USE_AVX2", False)
self._add_definition("USE_AVX512", False)

self._add_definition("USE_OPENMP", False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't with_openmp be used here or anywhere else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should, but before I need to be able pass tests at least for basic setup...

@conan-center-bot

This comment has been minimized.

@conan-center-bot

This comment has been minimized.

@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Oct 2, 2021

@fdgStilla thanks a lot for help, I'm testing all those changes locally rigth now. For some reason conan cannot resolve 'new' receipts line nsync and eigen thing to figureout what is going on ...

@conan-center-bot
Copy link
Collaborator

Failure in build 13 (2f2a35701ad2e70e95988e21e269d61b2c928151):

  • onnxruntime/1.7.1@:
    CI failed to create some packages (All logs)

    Logs for packageID 48aacb9b4b41ce25e6fc0d01b5e46b6ce1efcc4c:
    [settings]
    arch=x86_64
    arch_build=x86_64
    build_type=Debug
    compiler=gcc
    compiler.libcxx=libstdc++11
    compiler.version=5
    os=Linux
    os_build=Linux
    [options]
    onnxruntime:shared=False
    
    [...]
    Schema error: Trying to register schema with name ArrayFeatureExtractor (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 34, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 34
    
    Schema error: Trying to register schema with name Binarizer (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 58, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 58
    
    Schema error: Trying to register schema with name CastMap (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 115, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 115
    
    Schema error: Trying to register schema with name CategoryMapper (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 175, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 175
    
    Schema error: Trying to register schema with name DictVectorizer (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 236, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 236
    
    Schema error: Trying to register schema with name FeatureVectorizer (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 272, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 272
    
    Schema error: Trying to register schema with name Imputer (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 318, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 318
    
    Schema error: Trying to register schema with name LabelEncoder (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/old.cc line 61, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/old.cc line 61
    
    Schema error: Trying to register schema with name LinearClassifier (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 574, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 574
    
    Schema error: Trying to register schema with name LinearRegressor (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 622, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 622
    
    Schema error: Trying to register schema with name Normalizer (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 659, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 659
    
    Schema error: Trying to register schema with name OneHotEncoder (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 705, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 705
    
    Schema error: Trying to register schema with name SVMClassifier (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 813, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 813
    
    Schema error: Trying to register schema with name SVMRegressor (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 872, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 872
    
    Schema error: Trying to register schema with name Scaler (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 734, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 734
    
    Schema error: Trying to register schema with name TreeEnsembleClassifier (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1005, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1005
    
    Schema error: Trying to register schema with name TreeEnsembleRegressor (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1118, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1118
    
    Schema error: Trying to register schema with name ZipMap (domain: ai.onnx.ml version: 1) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1168, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 1168
    
    Schema error: Trying to register schema with name LabelEncoder (domain: ai.onnx.ml version: 2) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 461, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 461
    
    terminate called after throwing an instance of 'Ort::Exception'
      what():  Load model from squeezenet.onnx failed:This is an invalid model. Error in Node: : /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0/_/_/build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/schema.cc:877: SchemasRegisterer: Assertion `dbg_registered_schema_count == DbgOperatorSetTracker::Instance().GetCount()` failed: 4 schema were exposed from operator sets and automatically placed into the static registry.  413 were expected based on calls to registration macros. Operator set functions may need to be updated.
    Aborted
    eigen/3.4.0: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/eigen/3.4.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    eigen/3.4.0: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/eigen/3.4.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    nlohmann_json/3.9.1: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/nlohmann_json/3.9.1/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    optional-lite/3.4.0: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/optional-lite/3.4.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    optional-lite/3.4.0: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/optional-lite/3.4.0/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    safeint/3.0.26: WARN: Lib folder doesn't exist, can't collect libraries: /home/conan/w/BuildSingleReference/.conan/data/safeint/3.0.26/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib
    ERROR: onnxruntime/1.7.1 (test package): Error in test() method, line 25
    	self.run(bin_path, run_environment=True)
    	ConanException: Error 134 while executing bin/test_package
    

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Oct 3, 2021

@fdgStilla i have applied almost all your suggestions thanks again. For now there is two major issues:

  1. tons of errors like:

    Schema error: Trying to register schema with name LabelEncoder (domain: ai.onnx.ml version: 2) from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0///build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 461, but it is already registered from file /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0///build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/traditionalml/defs.cc line 461

  2. Tests failed for debug build:

    Load model from squeezenet.onnx failed:This is an invalid model. Error in Node: : /home/conan/w/BuildSingleReference/.conan/data/onnx/1.9.0///build/ed1e766999fbc4b1b5449f8be947b44e49b16bf0/source_subfolder/onnx/defs/schema.cc:877: SchemasRegisterer: Assertion dbg_registered_schema_count == DbgOperatorSetTracker::Instance().GetCount() failed: 4 schema were exposed from operator sets and automatically placed into the static registry. 413 were expected based on calls to registration macros. Operator set functions may need to be updated.

If you have any insights on those please let me know

@fdgStilla
Copy link
Contributor

@CAMOBAP I forgot I was not using your test_package, I used a very simplified test_package like this:

cmake_minimum_required(VERSION 3.1)
project(PackageTest CXX)

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
set(CMAKE_MODULE_PATH ${CMAKE_BINARY_DIR})

find_package(OnnxRuntime REQUIRED)

add_executable(onnxruntime_test onnxruntime_test.cpp)
target_link_libraries(onnxruntime_test OnnxRuntime::OnnxRuntime)
#include <onnxruntime_cxx_api.h>

#include <iostream>

int main()
{
    Ort::SessionOptions m_session_options;
    std::cout << "Success !" << std::endl;

    return 0;
}

I don't know why your test is failing, but maybe you can compare with mine and understand the issue.

@CAMOBAP
Copy link
Contributor Author

CAMOBAP commented Oct 4, 2021

@fdgStilla I have created a test program based on https://github.com/microsoft/onnxruntime/blob/master/csharp/test/Microsoft.ML.OnnxRuntime.EndToEndTests.Capi/CXX_Api_Sample.cpp

To me

Ort::SessionOptions m_session_options;
std::cout << "Success !" << std::endl;

Doesn't looks enough for such complex software as Onnxruntime (but maybe I'm wrong)

Looks I need to dig into the test program...

@fdgStilla
Copy link
Contributor

Doesn't looks enough for such complex software as Onnxruntime (but maybe I'm wrong)

I agree, I was juste a little lazy...

@fdgStilla fdgStilla mentioned this pull request Oct 11, 2021
@stale
Copy link

stale bot commented Nov 3, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 3, 2021
@stale
Copy link

stale bot commented Dec 3, 2021

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale stale bot closed this Dec 3, 2021
@gmeeker gmeeker mentioned this pull request Apr 1, 2023
3 tasks
@mayeut mayeut mentioned this pull request Apr 6, 2023
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants